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Abstract 

In order .to identify and dc^velop procedures for complying with the 
impact requirements of Public Law 94-482, PROJECT IMPACT studied 
five problems: (1) how to define impact, (2) how to assess impact,: (3) 
how to show cause and effect relationships between research and devel- 
opment (R&D) activities and changes in the vocational leuucation 
teaching-learning situation, (4) how to predict the probability of im- 
pact, and (4) how to facilitate the*impact of R & D activities. The 
methods used in these studies were to review and discuss views of 
experts and literature related to the problems and to analyze the pro- 
cess of impact in selected cases of R & D activities and exemplary 
programs in vocational education in Illinois. Findings of PROJECT 
IMPACT'S activities from August 1, 1978 to July 1, 1930 are reported in 
— nine-voltjmes-: — (-1^eoritext--ant}-Prrnclptes-i)f~A-^eSs^ 

Case Study of the Illinois Occupational Curriculum Project, (3) A Case 
Study of the Illinois Network of Exemplary Occupational Program for 
\ Handicapped and Disadvantaged Students, (4) A Case Study of Illinois 
' 'Vrojects in Horticulture, (5) A Case Study of Illinois Career Education 
Projects at the Awareness Level, (6) A Case Study of the Occupational 
Survival Skills Project, (7) Case Studies of Two Illinois .School Districts 
with Innovative Vocational Education Programs, (8) A Field Study of 
Predicting Impact of Research and Development Projects in Vocational 
and Technical Education, and (9) an Executive Summary. . ^. 

Volume"^ 9 provides at. overview of Project Impact as wsll as brief 
synopses of each of the case studies conducted within the two-year 
project. A summary of the reyiew of literature and the vi<3ws of ex- 
perts regarding impact is presented. The final section of the Executive 
Summary presents a series of generalizations and guidelines for assess- 
ing the impact of research and development activities in vocational 
education . 
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Chapter 1 
Overview of PROJECT IMPACT ' 

In 1974, the U.S. Office of Education (USOE) asked the 
jjatj pnal Acade my of Sciences (NAS) to assess the impact of 
approximately 250 million dollars spent by USOE^-^on vocational^ - 
education research and development (R&D) activities during the 
ten years between 1965 and 1974. The NAS Committee on Voca- 
tional Education Research and Development (COVERD 1976) re- 
ported that the R & D of the- deca de studie .d_bad_impact— on — - 
curriculum development but did not have documented, widespread 
impact on the knowledge, skills or employability of large numbers 
of students* National evaluations of vocational R & D in addition 
to the COVERD report have ^ been similarly critical (Development 
Associates, 1975; Rand Corporation, 1975; Comptroller General of 
the United States, 1974). Acting upon these reports. Congress 
passed the Educational 'Amendments of 1976 (Public Law 94-482) 
which mandated that contracts for R & vocational education 

not be allowable unless the applicant couid "demonstrate a reason- 
able probability that the contract would result in improved teach- 
ing techniques or curriculum materials that would be used in a 
substantial number of classrooms or other learning situations 
within five years after termination of such contracts" (Federal 
Register, 1977). 

PROJECT IMPACT is a state funded study designed to de- 
velop pror^idures for assessing the impact of vocational education 
research and development efforts on vocational education. The 



primary purpose of the study is to identif-^ and develop proce- 
dures for complying with the impact requirement of Public Law 
94-482. 

The study addresses itself to three problem areas: (1) how 
to define impact, (2) how to assess Impact, and. (3) how to show 
a cause-and -effect relationship between project activities and 
changes in the vocational education teaching-learning situations. 
The study also .addressed two subsidiary • problems: (1) how to 

» ■ ■ — 

_£i::edicjLJth how to manage on -going 

contracts to increase impact probability. 

The^ methods used to conduct tills study were to review 
literature related to the problem areas, to interview individuals 
with experience and expertise in the problem areas, and to ana- 
lyze the {process of impact of several programs of related projects 
which w^e funded by the Illinois Office of Education/Department 
of Adult, Vocational and Technical Education/Research and Devel- 
opment Section and one project funded by the Comprehensive 
Employment and Training Act (CETA). 

The major activity of PROJECT IMPACT has been to analyze 
the programs (cases) of related R&D projects. For the first 
year. It was decided to select two cases for "top-down" ' ^lalysis 
and two cases for "bottom-up" analysis (see Volume 1, Appendix 
A, p. 43). The project staff referred to these types of retro- 
spective analysis as "tracking." It was anticipated that the two 
types of tracking would produce different insights about impact. 
For example, bottom-up tracking might better identify "bottle- 
necks" to impact than would top-down tracking while top-down 



tracking might be more effective in relating project intents and 
, project outcomfes. One staff member was assigned as "tracking 
/ manager" for each of the four cases and was instructed to keep a 

detailed log of her/his activiites (e.g., identifying documents, 

Retrieving documents, identifying key people and interviewing 

them, recording data, anajyzing data). 

- Nominations for the first four case studies were sought from 

members . of PROJECT IMPACT'S Advisory Committee, project 

consultants, and project staff members. The'following cases were 

selected for top-down t.'-acklng: 

1. "A Research and Development Project in Occupational 
Education" (The Illinois Occupational Curriculum Project 
—1.0. CP.) which was developed by Joliet Junior 
College and funded by them and by the Illinois Depart- 
ment of Adult, Vocational and Technical' Education in 
fiscal years 1970-72. 

2. "The Illinois Network of Exemplary Occupational Pro- 
grams for Handicapped and Disadvantaged Students." 
At the time, the case study was intiated, the network 

. was in its fourth year of operation. it was in the 
"dissemination" stage and MOE/ DAVTE ' had funded 
\ Illinois State University to coordinate dissemination for 
the Network's eight demonstration projects. 
The third and fourth- cases, which were selected for bottom-up 
tracking, were: 

3. "Illinois Projects in Horticu I tore." lOE/DAVTE funded 
some twelve R&D projects in horticulture between 1967 



and 1978. Both* private and public schools participated 
• in these R & D. efforts. 
■ 4. ■ 'Jlllinois Career- Education Projects at the Awareness 
Level." iOE/DAVTE funded three qr more major pro-' 
jects in this area between 1970 and 1978, and CETA. 
began funding one for'the Illinois Department of Correc- 
tions in 1975. " . 

During its second- year PROJECT. IMP-ACT studied the "Occu- 

V 

pational Survival Skills Project" and "Two Illinois Sch^f Districts' 
with Innovative Vocational Education Programs" and developed a ' 
model for an "Impact Assessment System for the lllinos Board of 
Education/Division of Adult, Vocational and Technical Education/ 
Research and Development Section." ' 

PROJECT IMPACT'S activities for the 1979 and 198a fiscal 
years are reported jn nine volumes. Volume 1--Context and 
Principles of ^ Assessing lmf)act7-contains an introduction to 
PROJECT IMPACT, a review of literature and views of experts in 
regard to planned educational change and Impact, the methods • 
used to. Conduct impact case studies, a concise listing of refer- 
ences used during Phase 1 activities, and appended materials 
such as the initial project proposal and data'^coHection instruments 
which were devaloped for use in case studies. Supplemental^ 
reports of the project aro contained in the following volumes: 

Volume 2 - A .Case Study of the "Illinois Occupational Curricu- * 
- lum Project" ^ 

Volume 3 - ^ A Case Study of the "Illinois Network of Exemplary 
Occupational Programs for Handicapped and Disad- 
vantaged Students" , • 



A Case Study of "Illinois Projects in Horticulture" 

A Case Study of "Illinois Career Education Projects 
at the Awareness Level" " 

A Case Study of the "Occupational Survival Skills 
Project^' «^ . ' 

Case Studies of "Two lllipois Sphool Districts with 
-Innovative Vocational Education Programs" 

A Fie!d Study of "Predicting*'lmpact of Research 
,and Development Projects in Vocational and Techni- 
cal Education" 

Executive Summary of Volumes 1-8 and Conclusions 
and Recommendations for Assessing the Impact cf 
Vocational Education Research and Development on 
Vocational Educatfon . * 

■ ^ 
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Chapter 2: ' 
Summary of the Views 
of Experts and the Review of Literature . 
\x\ the fall months of 1978, PROJECT IMPACT'S staff- held 
several meetings with experts in problems related to assessing 
impact on social systems and conducted a review of literature on 
planned change, impact assessment and case study methodology. 
A summary of the ideas gained from these activities follows and is 
divided into three major sections: ' Planned Change, Impact Assess- 
ment and Case Study Methodology. It should be pointed out that 
these efforts were intended to serve as a springboard to the 

project and were not intended to be exhaustive of any of these 

« 

topics • , 

^ ■ ■ * 

Planned Change 

Watzlawick, Weakland and Fisch (1974) described two types 
of change: "first-order change" (where things have only JDeen 
m6ved around) anc^ "second-order change" (where a real differ-* 
ence jias been mdde). Methods of effectinti second-order change 
have been ' irivestigatecTas an area of study for about 20 years. 
The ^published work of a national conference on planned change 
held^at the University of Michigan in 1973 (Havelock & Havelock, 
1973) pre^nted four major perspectives as alternative models for 
effecting change: a Problem Solving Model, a Social lnter?ction 
ModeVr a Research and Development Model and a Linkage Model. 
Each of 'these models focuses on preadoption behavior* (awareness, 
interest, evaluation, trial and adoption), but ignores the issues 



of adaptation to an Jnnovative strategy. While a technology or a 
technological product is usually invariate in its implementation and 
in its outcome from one context to another, the implementation of 
an educational innqvation is yariate. The installation of the 
former is a mechanical process, while that of the latter is an 
evolutionary process, or what Berman & McLaughlin (1974) called 
a "mutatifn phenomenon." When an educational innovation is 
implemented, they pointed^ out, a two-way orgcnizational process 
of adaptation (mutual adaptation), occurs. \ 

Postadoption behavior of the school is one of the variables 
affecting student outcomes. Pincus (1974) suggested that project 
planners should consider the institutional change which any 
adoption would require 

Berman & McLaughlin (1974) proposed a new three-stage 
process model for planned educational change, a model which 
focuses on postadoption rather than preadoption. The three 
stagei are (1) a support stage, (2)" an implementation stage, and 
(3) an .incorporation stage. 

A considerable amount of literature on planned educational 
change focuses on adoption and implementation of products of 
educational R&D activities. Swanson (1976) pointed out that 
physical products are e&si1y recognized and understood, packaged 
and transp orted, whereas, intellectual products may only be 
recogni;zed and understood by the people who develop and use 
them. Consequently, intellectual products* are more difficult to 
package and trace* 
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Several studies have observed that measurable and/or observ- 
able changes (impact) which result from 8 & D activities are 
cumulative. It has also been observed that the best time to 
assess impact varies greatly. 

Impact Asses:;ment 
The review of literature and ideas on assessing the impact of 
educational R&D activities was intended to help answer the 
following questions: What is educational impact? How can it be 
recognized, measured, predicted and facilitated? 

What is impact and how can it be measured ? 

While Dick (1976) defined impact as measurable pheonmena - 
of positive or negative value - which follow the completion of a 
project or a program, Miher & Miller (1974) concluded that impact 
has at least two broad parameters - intended and actual impact. 
Both studies described similar methods of assessing impact. 

Stake (1978) and others concluded that much significant 
impact in education can be seen but not measured. Impact of an 
educational project can thuSo be considered as the measurable 
and/or observable changes/influences (positive or negative) that 
are intended, immediate and subsequent. These changes/influ- 
ences could be both quantitative as well as qualitative, and relate 
to the . process, products, inputs and environment of the educa- 
tional system to which the project is directed. Thus, impact 
studies of projects need a chain of assessment at the planning, 
formative, summative and subsequent stages. 
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Berman & McLaughlin (1974) stressed that it is necessary for 
evaluators to have data related to the conditions of the target 
group before and after the R & D activity. Dick (1976) advo- 
cated the use of two evaluators - one^ internal and one external to 
the project. 

How ccin impact be measured ? 

Swanson (1978) listed eight levels at which actual impact may 
be assessed - where educational impact occurs. These levels, are: 
national, state, regional, local, unit, departmental, classroom and 
community levels. Some tangible products of impact which may be 
measured and/or observed at these levels are changes in: capa- 
city for enrollment, actual -enrollment, completion, organization, 
attitudes, efficiency, attendance, status (of minorities), achieve- 
ment, knowledge, skills, cost, unemployment/underemployment. 

Three important strategies for measuring fmpact were identi- 
fied by var.ious researchers. O'Boyle (1974) pointed out the need 
for identifying unique criteria for measuring the impact of certain 
types' of prbjects. Henning (1975) pointed out that a certain 
research design has more impact on one kind of pr^-ect than 
another research design; this presupposes the choice of an appro - 
priate research design for a given project. 

The National Council on Employment Policy (1976) pointed out 
the need for assessing the duration of impact. They concluded 
that high initial cost of a project or program may be justified by 
the impact (gains)^ which remain constant over time. Thus assess- 
ment of actual impact should be divided into immediate" and 
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"subsequent" impact. This type of longitudinal impact assessment 
would require continuity in impact specifications for all state- 
funded projects (Cheney-Stern and Evans, 1979). 

How can impact be predicted ? 

Numerous characteristics which are associated with high 
versus low impact have been identified by researchers in voca- 
tional education and other fields. The degree to which these 
characteristics are present or absent may be useful in predicting 
actual impact. 

How can impact be facilitated ? 

Several studies have pointed to the heed for research pro- 
grams to form a coherent whole , a continuity between basic, 
applied and developmental research. Impact Is likely to be facili- 
tated if there is continuity of efforts on the part of researchers, 
continuity of goals and funding on the part of organization^ which 
fund research, and continuity of reporting successful research 
results to consumers. Copa (1978) suggested that impact might 
be facilitated if investment patterns matched ^the delivery system 
(i e. subject-matter rather than "across-the-board" projects). 

Case Study Methodology 
A case study is "the examination of an instance in action" 
(Walker, 1974), the purpose of which is to enable the researcher 
to make generalizations from the instance studied to the class it 
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purports to represent, to make generalizations from features of 
the ins&ce to a multiplicity of classes, or to make generaliza- 
tions about only the instance studied (Adelman, Jenkins and 
Kemmis, 1975). 

A variety of techniques as well as a variety of reporting 
forms are employed in the case study method. 

I 

Stake (1978) distinguished a case study project (the persis- 
tent study of a single case) from the multiple case study project 
•(a collection of individual case studies). The latter demands a 
form of linkage - "a manner in which to discuss their differences 
and similarities." 

The case study literature describes a validation technique 
called "traningulation" (Center for Instructional Research & Evalua- 
tion, 1978), the idea of supporting a controversial issue by at 
least three independent sources. A controversial finding, based 
on several accounts has more credibility than one based on only 
one accpunt. 

Anonymity of sites and persons and confidentiality of infor- 
mation are among the ethical practices which are common in case 
studies. These practices provide legal protection to case study 
researchers as well as privacy to individuals and institutions. 
Also clearance procedures .are often required at the time field 
contacts are made. 

»■ 

Crawford, Kratochvit and Wright (1972) at the American 
Institute for Research (AIR) developed criteria for case selection 
from a variety of eduational R&D activities, and six procedural 

I 

i 
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steps for multiple case study projects • They also suggested 
methods of data collection and methods of reporting the collected 
data. 

^ Other researchers have developed and used special forms for 
collecting data or series of matrices for analyzing impact. Miller 
& Miller (1974) compared intended versus actual impact, funding 
levels and types of problems for over 700 federal and state R&D 
projects in vocational education • 

After identifying and evaluating these and other available 
mutiple case study procedures and tools, the PROJECT IMPACT 
staff decided to adopt and adapt some of them forgathering and 
recording data on their first case studies which were conduced 
in 1978-79. Details of the methodology and samples of forms used 
for case nominations and data collection in the 1978-79 investiga- 
tions are contained in Volume 1 . Details .of the methodologies and 
samples of forms used in the 1979-80 investigations are included 
with the impact studies in volumes 6, 7 and 8. 
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-- • - CFapter T: 

- Summaries, Conclusions and Recommendations 
from the Case Studies 

During 1978-79, PROJECT IMPACT analyzed the Impact 
process of four groups of related R&D projects In vocational 
education (cases) which were funded by the State of Illinois. \^ 
during the past ten years. Two of these cases were tracked from, 
the top-down and two wei^e tracked from the bottom-up. The 
cases selected for top-down tracking were^v"The Illinois Occupa- 
tional Education Project" and "The Illinois Network of Exemplary ^ • 
Occupational Education Programs for Handicapped and Disadvan- 
taged Students". Since these cases' targeted primarily on post- 
secondary administrators and secondary teachers respectively, the 
- project staff decided that selection criteria for the bottohi-up 
cases should be that they targeted on students or teachers and 
students; also that one of these cases should be a subject-matter 
specialty rather than an "across the board area". The two cases 
chosen for bottom-up tracking were "lllmols Projects In Horticul- 
ture" and "Illinois Career Education Projects at the Awareness 
Level". The details of the methods and data collection Instru- 
ments used for these four case studies are contained In Volume 1. 

Case studies added during 1979-80 used case study methods^ 
such as document analysis, site visits. Interviews and question- 
naires but did not Involve top-down or bottom-up tracking. 
These new studies were: an impact assessment of ♦le "Occupa- 
tional Survival Skills Project" and "Two llli.riois School Districts ^ 
vyith Innovative Vocational Education Programs". PROJECT IMPACT 

19 
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also continued to monitor the impact of R & D activities for improv- 
ing vocational programs for handicapped and disadvantaged stu- 
dents. ' ' 

In addition to these case studies, a field study "Predicting 
Impact of R & D Projects in Vocational and Technical Education" 
was conducted by a research fellow assigned to PROJECT IMPACT 
in 1979. 

Following are summaries of the findings, conclusions and 
recommendations from each* of these impact studies. 

The Illinois Occupational Curriculum Project 
The Illinois Occupational Curriculum Project (lOCP) was 
actually a group of related R&D projects which were funded in 
1970-74 by the Department of Adult, Vocational and Technical 
Education. The Research and Development Section sponsored the 
1970-72 activities^ and the Curriculum and Personnel Development 
Section sponsored the 1973-74 activities of demonstration and 
initial dissemination. The innovations lOCP introduced were 
intellectual processes and physical products (resource manuals) 
for developing occupational programs and courses. Since 1974, 
the resource manuals have been disseminated through a commercial 
publisher. The title of the publication is Plahning, fmplementing 
and Evaluating Career Preparation Programs (Borgen & Davis, 
1974). The group?; targeted for impact from lOCp were vocational 
education administrators from local and state education acj'^ncies 
(LEAs and SEAs). 

2y • ■ 
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Local and State funding for lOCP's R&D activities spon 
sored by the Research and Development Section totaled 227,297 
dollars. PROJECT IMPACT was unable to retrieve information on 
the amount of funding lOCP received from Illinois for demonstra- 
tion and dissemination workshops. Several other states sponsored 
similar workshops. Prior to the avaflability of the commercial 
publication, the State of Illinois and the lOCP developers dissemin- 
ated approximately 3,500 sets of the manuals. According to the 
publisher's records for 1974-1979, an additional 1,758 were dissem- 
inated throughout nearly every state and territory of the United 
States. 

In developing procedures to assess lOCP's impact within five 
years after State funding of the project was terminated, PROJECT 
IMPACT limited its investigation to an intended impact on an 
intended target group— to improve the curriculum development 
practices of the approximately 500 vocational education administra- 
tors who attended the lOCP workshops in Illinois. Such retro- 
spective assessments have usually failed to show Inpact because 
efforts to retrieve project proposals and reports were unsuccess- 

^fulL lOCP documents were identified from xoncise listings in 
lOCP reports. Documents were retrieved from a variety of 
sources such as DAVTE, lOCP staff, lOCP consultants and the 
ERIC collection. Thorough document analysis paid-off in finding 

^a^^fine data on 1971 curriculum development practices in Illinois 
commLmHy^ These 1971 data verified 't^e underlying 

assumptions lOCP and pinpointed practices which needed the 
most improvementN Assessment of I OCR's impact was facilitated 
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by our being able to replicate the 1971 survey in 1979. We found 
that many curriculum development practices changed (improved) 
from 1971 to 1979 and that 'many of these changes were significant 
at the .05 level. Because of the impossibility of controlling 
intervening variables, these changes can not be entirely attri- 
buted to lOCP. However, lOCP was a major treatment variable 
during this time in Illinois and we feel confident' in saying that it 
did achieve its intended impact on an ip:end'"d target population. 
The lOCP case study demonstrated the value of funding a re- 
search project to thoroughly investigate a problem and gather 
baseline data. It also demonstrated the value of maintaining files 
for documents on completed projects. A pitfall common to impact 
assessment which lOCP did experience was neglecting to obtain 
complete names and addresses on workshop rosters. Olher charac- 
teristics of lOCP which may bear on its impact are: 

• The proposal for the project was In r-jsponse to a felt 
need at the local level and not in response to a Request 
for a Proposal (RFP) from an official funding agency. 

• The project's developers and their institution had good 
track records in R & D activities. 

• The project was thoroughly pJanned and the felt need 
for the project was verified with baseline data on the 
group and the behaviors targeted for impact* 

• The processes and products of the project were care- 
fully developed, pilot-testeci, field-tested and contin- 
ually evaluated. 

22 



• .The project had continuity- in funding and in staff. 

• ' The project hired external evaluators as consultants. 

The Illinois Network of Exemplary Occupational Education Programs 
for Handicapped and Disadvantaged Students (The Network) 
PROJECT IMPACT'S top-down tracking of the Network, began 
In FY 79--the last year that the Network received R&D funds. 
Tracking was continued during FY 80 to assess the impact of 
discontinued funding on the overall Jmpact of the Network, The 
Network was funded by the lOE/DAVTE for four years at a totaf 
cost of 1.6 million dollars. The purpose of the Network was to 
- expand the quality and quantity of special needs vocation?*! educa- 
tion in the State of Illinois. Originally the Network was com- 
prised of nine sites. One of the sites was centrally located and 
was devoted to dissemination and coordination of tjne Network's* 
efforts. Other sites which existed throughout the state consisted 
of one community college, one area vocational center and six 
comprehenesive high scFtools. In addition » to providing Illinois 
educators with accessible programs for on-site viewing, the. 
Network also served as a source of materials, processes, consult- 
ed ing services and in-service training appropriate for providing 
vocational education for students with speci^al needs. 

. At the inception of the Network in 1975, the variability of 
program and process development at all sites reflected the individ- 
ual problems and available resources within the site district. The 
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qategories of the products developed were: administration, com- 
munity resources, curriculum, support services, inservice, special 
techniques, special programs and individualized planning. There 
was reluctance at the central dissemination site to push for use of 
Networ|< products because there were no data on the effectiveness 
of most of the materials • Very— few products were formally field 
tested within the Kietwork* Consequently, materials developed 
elsewhcire were adopted, adapted and disseminated by the Net- 
work • ' . 

Processes of evaluation >vhich proved to have been important 
to the Network were: internal evaluation at each site, third- 
party (external) evaluation at each site, and the annual submls- 
jBion of a continuation proposal and. final report* 

During its thira and fourth years, the Network's emphasfs 

o * 

was on State-wide diffusion of mainstreaming materials. ' One 
highlight of the 1977-78 schodi year dissemination activities was a 
two-day, state-wide conference held in Decatur during November 
1977. Over 200 educator? from Illinois attended the conference. 
Other states were also represented. 

Also during the 1977-78 school, year, the Network printed 
and disseminated booklets which described Network activities by 
site. Conference and workshop presentations in the state and 
nationally at the annual meeting of the American Vocational Associ- 
ation, and quarterly newsletters on special needs vocational 
education were among the additional 1977-78 school year dissemina- 
tion activities. Similar state-wide conferences and one=-day work- 
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shops were held during the 1978-79 school year and were well 
attended . 

The Network enjoyed a high level of visibility both within 
Illinois and, to a certain extent, nationally among educators 
interested ft^ special needs vocational education • A perusal of 
workshop evaluation forms and letters requesting information on 
file at th^j^llinois State University dissem'ination site bears wit- 
ness to the Network's ability to reach many educators. In quaHta- 
tive terms, there is little doubV that the Network achieved the 
impacts of increased awareness and increased acceptance of the 

^ 

need for mainstreaming special needs students in vocational pro- 
grams. In 1979,'*' PROJECT IMPACT recommended that the 
Network collect quantitative baseline and follow-up data on these 
intended impacts and that both the Network and PROJECT 
IMPACT should attempt to collect qualitative and quantitative data 
on increasing enrollments of special needs students in vocational 
programs. 

In FY 80; PROJECT IMPACT contii;Kied to monitor the pro-' 
gress of the hjetwork sites and specifically examined the impact of 
discontinued funding. A foHow-up questionnaire was mailed to 
the former project directors and principal staff members at each 
of the Network sites. In addition, two interview sessions were 
held with these individuals in May 1980 to. further explore and 
document Network activities during 1979-80. This follow-up study 
focused on three major questions: (1) To What extent have the 
programs and services for special needs students been expanded, 
maintained, or reduced? (2) How have staffing and student 
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enrollment patterns changed? (3) ' To what extent does each 

• . A 

district (Network site) continue to provide Inservice and .assis- 
taace to others? . ' o 

The major observations noted in the 1979-80 follcfW-up impact 

study were as follows: _ . 

• t- . • 

* 1. In ail cases the H & D 'Network funding^ had led to the 
s> enhancement and expansion of special "^€ed^ programs 
and suipport services for special needs .students 'witfjin 
the funded districts. ^ • ' 

2. With only one exception, the programs and 'services 
established" wi:}»'the project funding had been main- 
tained dtiring 1979-80 at the seven sites participating in 
the follow-up study. $ 

3. Generally, the sites Reporting in the fpjjow-up study 
indicated that the number of handicapped and disadvan- 
taged students served in vocational_education programs 
had increased over* previous years^. [ 

4. During 1979-80 the NetWork sites reported several 



instances of communicatioo with other districts for the 
purpose of adopting or adaptmg. programming models 
developed at the Network site. 
5. ' The' success of efforts to continue working with selected 
• V "satellite sites" appeared to have varied results. Three 
Network sites had continued to be extensively in^volved 
- with their satellite sites in 1979-80. 



6.- As a result of sCibstantial staff turnover several Network 
sites reported a continuing need' for inservice staff 
development activities within their 'own' districts. ; 
Some characteristics of the Network which may bear on its 
fmpact are:*'" ' ^ - . 

— The^ project \^^^ in regppnse to Federal legislation 
rather than a felt? need at the local level 
the project staff members were not experienced in R & 
D activities 

There was a high rate of staff turnover at several of 
the Network sites - 
The project did not collect baseline data on its target 
groups to verify its assumptions about them 
The project did not formally field test its products 
The project was externally as well , as internally evalu- 

__ated , ' ' ^ 

The prbject had continuity in funding 
Thte project did not- formally collect pre or post data on 
workshop participants/ in terfns. of -their attitudes, 
kno^/teclgip oV skills. 
- • ' In. reflecting on. assessing . the;, impact of the Network, 
-PROUI^T IMPACT concluded^that trial use of the State's data 
bank might be worthwhile; for example, the nurubers of H & D 
enrollments claimed for Reimbursements by the . vocational programs 
at. the nine Network sites could be compared for 1975 (the year 
the Network began); and 1980-84 (the five-year period from the 
time the Network's vfundihg was terminated). If enrollment data 
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from nine comparable sites could be collected for the same years, 
the two sets data could be compared to see if the experimental 
sites have hsd. greater increases in H & D enrollments in voca- 
tional programs. ^ ^ 



Illinois Projects ilT*Horticulture 



The horticultureba^^tudy was an attempt to develop proce- 
dures for assessing the impact of a group of many (but small) 
related projects. Unlike the lOCP and Network case studies 
^which were conducted fhom the top-down (developers of products 

to users of products), the horticulture case study was from the 

\ 

bottom-up (intended users to developers). Fourteen projects 
costing about $200,000 over ten years were identified. A case 
researcher familiarized himself with as many of tbe R&D pro- 
ducts he could find from ^these projects. However, PROJECT 
impact's '"bottom-up" schem^^ was compromised when the case 



Researcher was^ unable to find many of the products and had to 
gq to the developers for assistance. Use of these products was 
then looked for in a sample of horticulture programs in Illinois. 
One of the major findings of this investigation was that users 
were often unaware of the origins of the products they were 
using. Parts of many products were found to have been passed 
down through the ranks of "hort" teachers (this was found to be 
the case with administrators in. the lOCP case study too). It was 
.also learned that most "ho^t" teachers are "ag" teachers and that 
"ag" teachers have an annual turnover rate of 10 percent or 
mc!r4. It was conluded that good "hort" products (or revisions of 
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good products) need to be re-disseminated every few years. The 
information on turnover rates raised the question, "What is known 
about, turnover rates for teachers in other vo-tech areas?" It 

was also learned that the "hort" teachers heard about instruc- 

f 

tional materials and workshops through established communciation 
channels with colleges of agriculture rather than through educa- 
tion a l R & D link a ges. — Th e r. t:tf nb e r$ of horticultur e pr ogi^ms, 
teachers and students in Illinois were found to have increased 
greatly during the 10-year span of R & D projects analyzed. Had 
the characteristics of "hort" teachers been known and utilized, 
dissemination practices would have (no doubt) differed and the 
impact of these related projects would (no doubt) have been 
greater.^ The major conclusions of this case study were that 
bottom-up researching was valuable in revealing and/or verifying 
characteristics of teachers which were relevant to facilitating 
impact. It was also learned that one procedure an impact study 
should not rely on is: asking teachers if they do or do not know 
about (or use) a certain product, in many instances they are 
using it and don't know it. 

Some characteristics of the horticulture projects which may 
bear on their impact are: 

• The origins of projects came from suggestions at meet- 
ings and RFPs. There was no planned program of * 
projects until the last few years. 

• Most projects were narrow in scope. 

• Most projects were low-cost, low-risk. 
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« Most products developed by projects were not formally 
field-tested and revised before being disseminated . 

• Most products developed by projects were not widely 
disseminated* 

s Projects did not study the characteristics of their 
intended target groups. 



Illinois Career Education Projects 



"aFthe Awareness Level 



0- 



"Career Awareness" projects for grade-school children and 
for incarcerated adults constituted this bottom-up case study. 
Findings are summarized for each gr^up--elementary school popu- 
4ations^,and__pnspnLpppulations. ^ . 



Elementary School Populations 
PROJECT IMPACT identified three major career awareness 
projects f^r children which were funded by the State of Illinois 
gyring the Seventies. These projects were: The Career Develop- 
rr^ent Project for Children (CDCP)/ Project ABLE and Project 
0|:CUPACS. A case researcher familiarized himself with the 

products of these projects and then set out to see to what extent 

I ' - ^ ' 

t ♦ 
the products were being used by elementary level schbol teachers 

InLiillO-Ois. — Because of . the large size- of this- teacher^ group, 



PROJECT IMPACT decided to arbitrarily look in depth at a few 
districts (large, medium and small) rather than to draw a random- 
sample of elementary teachers for the entire State. Based upon 
this very small sample, PROJECT IMPACT found that elementary 
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teachers were presenting career awareness content but were using 
very old (or no) instructional materials. The teachers in the 
sample were not using any of the products of these three major 
projects. Furthermore, none of these teachers had even heard of 
the projects or their products. Obviously, the sheer size of this 
group of teachers in Illinois ahd throughout the nation calls for 

special dissemination considerations. ^ 

^-^J^ 

At this point in the case study, PROJECT IMPACT decided 
^o^'continue the bottom-up tracking of only one . of the three 
projects for children— the Career Development for Children Project 
(CDCP). The CDCP was funded February 1970 - August 1973 at 
a cost of $272,786. It was funded just prior to USOE's big push 
for career education, and it impacted at the national level before 
it ever reached its original intended target group of classroom 
teachers. From this case~lUJay7TWWEeT-i^ 
of the need to monitor changes of the group(s) intended for 
Impact. Otherwise, in efforts to assess the specified impact at 
the-peak time, one-may-end upHooking-at the-wrong-groupT— 

Because of the national interest in CDCP, the State made a 
decision to solicit bids from commercial publishers for the dissemin- 
ation of CDCP. Only one bid was received and it was from an 
established firm that had experience with teachers of industrial 
arts and technical-career subjects but hot teacheFs of children in 
the elementary grades. It seems that the State, the publishers of 
CDCP and PROJECT IMPACT have all been niave about the conse- 
quences of entering new turf! Certainly, more thorough Investi- 
gations of new intended target groups are required in order to 
facilitate the impact upon them. 

31 ■ 
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In the days since tracking of CDCP was completed, PROJECT 
impact's staff members have continued to ponder and investigate 
ways of assessing the impacts of the investments in career educa- 
tion during the last decade and ways to facilitate impacts not yet 
realized from these investments* Two possible means have 
emerged to date. At the beginning of the decade and at the end 
of the decade, the United States Department of Labor funded 
Na.tipnal_LpngitudinaL .Surveys of the labor farcer ln"both sur- 
veys, thousands of young men and young women were interviewed 
about their knowledge of spetific careers. The interview ques- 
tions were nearly identical for each randomly selected sample in 
each survey. If the average scores from the first NLS were 
compared to scores from the second NLS, it would provide a 
measure of the impact of career^ education. It might be possible 
to obtain sub-sets of scores for Illinois youth in the samples and 
thereby ma Ke ^addrtiorrai-comparisons^ 

During the past year PROJECT IMPACT has studied various 
ly.pes_-OjLaDDLUaLcep_Ojrts._d one b y vocational pro grams in the State 
One of the formats used by Area Vocational Centers (AVCs) is to 
show the total number of enrollees for each participating school 
district and then to show what proportion of each district's high 

school population that figure repr esent s. ^In reviewing several 

annual reports of this nature from the same AVC it was learned 
that there had been steady increases in the proportions of high 
school students who enrolled at the AVC* If these types of data 
were compiled from all the AVCs' annual reports to the State, it 
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would seem to be another good measure of career education's 
impact. 

Iri^ contemplating pay offs from past as well as future invest- 
ments jn. career education, it seems that a large number of career 
education materials have been well planned and well developed but 
poorly disseminated. As more is learned about dissemination and 
utilization , it seems that it migh t be more wor tjTwhih--tOL-r-evlse- 
and/or reprint some of the materials already develop^ than to 
invest in the, planning and development of additional career educa- 
tion materials. 

Prison Populations . 

In the past, most evaluations of correctional programs have 
been long-term and most- of these have reported impact- in terms 
of recidivism. Recently, however, researchers in correctional 
education have concluded that if research and evaluartiorT of correc- 
tlonal-caeeecediicati on are to nay off, criteria other t han program 
effects on recidivism must be established. They would like to see 
more immediate (formative) evaluations of the impacts of compon- 
ents of career education programs. PROJECT IMPACT retrieved 
such data while Investigating the Career Orientation and Aware- 
ness Program (CO'A'f^'. 

COAP is an experimental prevocatlonal project for prisoners 
which was first funded by CETA in 1975 at a cost of approxi- 
mately $85,000 a year. One of the Irrtended Impacts of COAP is 
to facilitate vocational maturation of recently incarcerated young 
men. One of COAP!s assumptions Is that If prisoners develop a 
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more accurate self-concept, they will make better decisions about 
choosing a career and preparing for it. 

Among the evaluations of COAP was an impact study to see 
if the experimental program increased the accuracy of prisoners* 
self-concepts in terms of vocational abilities ?^d vocational needs 

(Cheney-Stern, 1977). A_1976_studeDi-9 

non-student group of prisoners were tested before and after 
COAP. It was found that both groups* estimates became more 
accurate but that some of the gains for the student group were 
significantly grpater.S- 

Other .data on actual and prospective COAP students which 
were collected and analyzed provided some insights into facilitat- 
ing rather than assessing impact. When prisoners' profiles of 
vocationai needs were compared to reinforcer profiles of occupa-i^\j' 
tions, it was found that many goods-ser\4cing and selling occupa- 
tions were most likely to satisfy the prisoners but that many 
construction and manufacturing occupations were not (Cheney- 
Stern, 1978). A' list of the occupations wh1^:trnwrere-Trrost~ftkety-^ — 
satisfy the (actual and prospective) COAP group was then com- 
piled and compared to a list of occupations which prisoners (in an 
earlier study of essentially the same population by Galloway in 
1972) had indicated the most interest in receiving training. The 
occupations of highest interest (auto mechanic, construction 
trades and heavy-equipment operator) were among those least 
likely to satisfy. This finding would seem to indicate that pro- 
grams based upon prisoners' inventoried needs would differ 

* • 
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markedly from programs based upon prisoners' expressed inter- 
ests • Would feedback on standardized tests (inventories) of 
abilities and needs re-direct prisoners interests? And would 
correctional (or regular) vocational programs based upon student 
and manpower needs be more successful than ones based on 
student interests and manpower needs? 

In addition to looking at lliinois-funded career awareness 

) 

projects; PROJECT IMPACT reviewed numerous Abstracts of 
evaluative studies on career awareness (prev^ational) programs • 
Although a tally was not kept, it seemed that few courses or 
-programs of less than 80 hours had any impact. Courses or 
programs of 80 hours or more were much more likely to produce a 
significant difference— especially if learning activities included 
hands-on experiences. During its first year (1976), COAP was 
an 80-hour course ^hich did include some hands-on activities • 
PROJECT IMPACT concluded that characteristics associated with 
impact of career education programs deserve furtneP investigation 
so that the ^knowledge gained can be incorporated into curriculum 
guidelines or standards for career education programs at orienta- 
tion, awareness and exploratory leve^u- 

The Occupational Survival Skills Project 
The Occupational Survival Skills (OSS) project was func'ed 
by the State from .1978 through 1980 at a total cost of about 
$90,300, Instead of tracking the OSS project from the top-down 
or bottom-up, PROJECT IMPACT decided to involve former OSS 
staff in assessing the impact of their own project. By this point 
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in time, OSS was one of the selected projects"'to"be disseminated 
nationally by the National Center^ for Vocational Education Re- 
search • Documentation cf nation-wide impact of OSS was largely 
accomplished by retrieving and re viewing files of correspondence. 
Fortunately, these files were still being retained by the devel- 
opers of the OSS project. Many qualitative statements from OSS 
users (teachers and students)) in various types of learning situa- 
tions were compiled from thev^ files and presented in PROJECT 
impact's case study* / 

In retrospect, it seems ihat such efforts and such results 
are in perfect compliance with P.L. 94-482. To go beyond this 
level of data collection is 'not required by law. However, many 
impact researchers have concluded that impact assessments should 

include quantitative data whicf;^ show that a measurable difference 

/ 

has or has not been made wi,thin a given time frame of the intro- 

f ' ^ 

duction of an innovation into^the educationat system. 

} 

Types of quantitative ?data which were mentioned but not 
measured In the OSS Case study were factors such as decreased 
rates in st udents-^dropping out of_schooLand„decneased„ rates In 
students leaving (or being asked to leave) their training stations 
in^ cooperative vocational education programs* Certainly, It 
should be" pbssible^tp obtalr^ such data for schools using OSS and 

comparable schoiols not using OSS. However, if comparisons were 

J 

made and differences were found. It would be difficult to attri- 
bute these differences to the use or non-use cf OSS materials 
because of other possible treatment or intervening variables which 
were operating at the same time. Nevertheless, the comparative 
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data could be useful in making judgments about the impact of the 
OSS project. 

One of the OSS' staff members who participated in PROJECT 
IMPACT'S study of OSS stated that If the OSS staff had known 
they were to conduct an' impact assessment on themselves, they 
would have been more formalistic with the testimonial data they 
received about OSS. For ^i-ample, they would have solicited 
testimonials by providing a tear-out, fill-out and send-back 
evaluation sheet in every volume of OSS materials disseminated. 

Two School Districts with Innovative 
Vocational Education Programs 
During PROJECT IMPACT'S second year of activities a deci- 
sion was made to study the process of vocational educations 
impact by examining two Illinois communities where the school 
districts' vocational programs were considered to be the best in 
the State, 

To guide these^ community studies, a set of questions having 
to do with five major educational issue areas was developed. 
These five issue areas were: sex equity, special needs students, 
career education, basic skills, and entrepreneurship (self- 

Q 

employment). 

The initial phases of these two case studies focused on 
developing a comprehensive view of the communities, the schools 
and the vocational education programs^ Extensive interviews were 
conducted with citizens in each of the communities representing: 
business/ industry, labor, governmental agencies, local news 

37 
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media, parents of school-age children, and various civic groups. 
In addition, numerous school personnel, including school board 
members, administrators, staff and students were interviewed at 
both locations. Numerous documents describing the communities, 
schools, and vocational education programs were also obtained and 
analyzed. Finally, two surveys were conducted to assess the 
perceptions of community members and school personnel regarding 
needed program improvements for their vocational education pro- 
grams. 

V 

Each of the communities and programs studiec! have numerous 
unique characteristics which necessitated a careful examination of 
the programs on an individual basis. Each program appears to be 
well-tuned and responsive to community^ economic needs, as well 
as the career interests and needs of students and parents ^at the 
secondary level. Various hypotheses were posited and discussed 
in an attempt to describe the reasons for the existence of effec- 
tive vocational programs. These hypotheses include the notion 
that selected administrators, with a backgrouncj ln_vocat}onah 
education and ^--^holding ex.ended tenure in the community may be 
the reasbn"*~for a successful program. Another major hypothesis 
discussed the homogeneity of community values. One of th3 
programs studied operates quite effectively in a small., rural, 
.agricultural community where the cultural values seem to be 
homogeneous and highly cr .sistent with the objectives of voca- 
tional education. Another hypotheses postulated a high positive 
correlation with the quality of the instructional staff for the 
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vocational education program. A fourth hypothesis noted a possi- 
ble relationship to the perceived value of vocational education 
programs as held by academic teachers and counselors. As 
vocational education continues to look at its impact upon a com- 
munity, a large number of reasons can be cited for it's effective- 
ness or Ineffectiveness. More comprehensive methods and 
strategies for conducting in-depth analyses of vocational education 
in a community setting are clearly needed. Fijlure research 
should focus on helping vocational educators develop or adapt 
tools for collecting and using ethnographic data and information. 
Such tools would be extremely helpful in helping to better under- 
stand how vocational education works for (impacts uptfn) individ- 
ual communities and students. 

/ 

I 

The community members 'and school personnel were surveyed 
to determine their perceptions of how well the vocational education^ 
programs responded to each of the major—rssues^' or^ problems. 
Across- both xommunities it appeared that a major concern existed 
relative-to the role of vocational education in improving the basic 
skills (writing, reading, and communication) of high school gradu- 
- ates.. AJI groups felt this was the most important issue for voca- 
tional education *■ to address in a more effective manner in the 
future. ^ The least important issue for future focus, from among 
the five listed in the survey, was self-employment or ert- jpre- 
neurship skills. 'v 

Predicting Impact of R & D Projects 
A research fellow assigned to PROJECT IMPACT for the first 

if 

seven months of 1979 investigated project characteristics consid- 
ered to be most important in the success of "research" versus 

ERIC . 
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"development" versus "research and development" Aactivitles In 
vocational and technical education. Findings from the study 
indicate that most characteristics which are important to success 
of "research" are also important to the success, of "development" 
or "research and ' development" activities. As a result of the 
investigation and discussions with State staff, PROJECT JMPACT 
concluded that decisions about funding projects and the impact of 
the projects funded would be facilitated if funding proposals were 
evaluated? on a form for rating the extent to which important 
characteristics of a project are present at the tim^' of funding.,. 
Such an instrument was developed but not pilot tested by PROJEjCT 
IMPACT. Use of such a rating form may be further investigated 
by PROJECT IMPACT and the Research and Development Section 
of DAVTE during FY 81. 
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Chapter 4: 

Generalizations and Guidelines Regarding the Impact 
of R & D Activities in Education 
Literature reviews, discussions and case studies have helped 

« 

PROJECT IMPACT'S staff to draw some generalizations and guide- 
lines regarding the Impact of R & D activities in education! 
These are: . . 



• 

1. Research programs should have a coherent whole In 
order to have Impact. There should be continuity 
between b^slc, - applied and developmental research. 

—2^. lmpacts_ape-measupablfr-and/or^observable-€hanges^-in 

specific condltk)ns and/or behaviors of specific popula- 
tions. . > 

3. Impacts are cumulative and the best times to assess 
. them vary greatly. Some Impacts are Immediate and 

.some are subsequent. Some are sustained and some are 
not. 

4. Impatt and Its assessment are facilitated to' the extent 
that formative evaluation Is carried ,out and that base- 
line data on conditions and behaviors of target groups 
are available. 

5. An external (as well as an internal) evaluator should be 
employed throughout a project (or program of related 
projects). 

6. Intended Impacts and/of' populations targeted for Impact 
frequently change during the life of R & D projects. 
Therefore, Impact specifications n6ed.to be revised and 

t> monitored throughout a project. 



'40 

7. Retrospective assessment of impact should not be at- 
tempted , unless planning papers and progress reports 
^re available for thorough document analysis* 

8. The adoption process is different for a technological 
product (e*g. a calculator) than for an educational 
innovation (e.g* a new math curricuium). Educational 
R&D products may be physrc^ or intellectual. Physi- 
cal products are easily recognized, understood, pack-, 
aged, and transported. Intellectual products may only 
be recognized and understood by the people who de- 
velop and use them and they are more difficult to 
package and trace. 

9. Planned educational change should Vocus on the schooPs 
postadoption behaviors ^(support; -implementation and 
incorporation) rather than its preadopt'ion behaviors 
(awareness, interest, evaluation, trial and adoption). 

Some of PROJECT IMPACT'S conclusions and rjscommendations 
have resulted in guidelines which pertain more to directors of R & 
V projects and some which pertain more to funding agencies.- , 

Guidelines for Project Directors 

1. Design a research program of linked projects where a 
problem is identified and clariiied' before a solution to it 

^ is developed, tested and disseminated. 

2. Employ an ^external evaluator who can examine raw data 
for accuracy, report to the funding agency^ hold dis- 
cussions with staff and/or the target population, help 
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identify problems and informally offer alternative solu- 
tions. 

/ 3. Specify intended and actual impacts in qualitative (ob- 
servable) and/or quantitative (measurable) terms and 
revise them as necessary. 

4. Specify the population(s) to be impacted and revise 
these specifications as necessary. 

5. Specify the methods and tools to be used for assessing 
baseline data in the problem-identification stage of a 
project. 

6. Suggest methods and tools to be used for observing or 
measuring changes from baseline levels. 

7. Estimate the time frame for peak impact. ^ 

8. Maintain a record of previous research which is used as 
a resource. A concise listing of published and unpub- 
lished resources helps to document the impact of basic 
research on applied research-; ^-^ 

Guidelines for. Funding Agencies \ 

1. Fund projects which are part of a coherent whole or 
which (fan be expected to be p^rt of a future set of 
projects. 

2. Maintain archives of project documents such as planning 
papers, proposals, progress reports and final reports. 

3. Develop an impact specification format which can be 
coded for storage and retrieval on computers or micro- 
forms. 
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4. Require impact specifications for each project (or pro- 
gram of linked projects). This can (and probably 
should) be done at the RFP stage. Writers of unsolic- 
ited proposals should prepare their own specifications. 

5. Require baseline data of each R&D project (or p^- 
gram of linked projects), if these data "are not In the 
RFP or the proposal, require collection- of. _ tbem as a 

6. Monitor impact specifications/revisions throughout the 
^ life of a project (or program of linked projects). 

7. Assess the impacts of major completed projects in accor- 
dance with the baseline data and estimates of peak 
impact given by the former project directors. 

In order to apply these generalizations and guidelines for 
achieving impact and assessing it, PROJECT IMPACT has worked 
with the Research and Development Section (RDS) to develop an 
"j"!)?!?!?-?.!!*^]^'® model" of an impact assess.ment^system. — -Step-by— 
step procedures have been developed and are to be refined and 
instaHed over the next fiscal year. These procedures are sum- 
marized in a flow chart. (See Fig. 1). 
J- 

, These procedures will be reviewed and refined with the staff 
members who will be implementing the impact assessment system 

s 

over the next fiscal year. Once the system is installed, proce- 
dures 1 and 2 may be omitted for as long as the classification 
system and criteria for ultimate impacts are found to be satisfac- 
tory. In a separate document, each step is described in narrative 
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form" and has an activity sheet to go with it which specifies 
responsibilities, decision criteria and resources' for personnel.* 



> 




♦Proposed procedures for installing an impact assessment system for the 
Research and Development Section of the Illinois Board of Education, 
Department of Adult, Vocational and Technical Education. Unpublished 
manuscript prepared by PROJECT IMPACT, July, 1980. 
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1. 



The RDS adopts, 2, 
adapts or develops 
a classification ^ 
system for R & D 
projects. 



7. 



The contract 
administrators 
develop impact / 
specifications 
and determine 
adequacy of 
baseline data for 
projects. 

The contract 
administrators 



5. 



The RDS develops 
impact criteria 
for each class 
or sub-class. 



The contract 
administrators 
prepare and 



The RDS classifies 
its current 
projects. 



maintain projecV 
files . - 



The manager 
of the RDS 
assigns staff 
to projects as 
-contract 



8. 



develop planning, 
packets of materials. 



12. The RDS reviews 11. 
projects which / 

are expected tot.^ ^ 
peak during next 
FY, selects pro- 
jects for impact 
assessment and 
collects follow-up 
data. 

13. The contract 14.\ 
administrators 
compare baseline v 
and' follow-up ' 
data reports impact 
findings and 

closes files: 



The contract 
administrators 
develop ,RFPs. 



Contract administra- 
tors file impact ^ 



9. 



10. 



monrtoring forms 



administrators. 



The RDS reviews 
proposals and 
makes contracts 
to have projects 
carri|d out. 

Project staff 
members and ^ 
contr3ct 
administrators 
complete impact 
monitoring forms. 



The RDS reviews, 
compiles, discusses 
and utilizes impact _ 
findings in planning 
R & D activities. 



Fig. 1 An Impact Assessment System for 

a Research and Development Section^ (RDS) 



4G 
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As the Research and Development Section and PROJECT 
IMPACT work together to refine and implement an impact assess- 
ment system, they will attempt to identify furth"er~giJiddirTir-fdr 
facilitating the impact of R & D activities. It is anticipated that 
these efforts will result in a "handbook" for achieving and assess- 
ing R & D impact. 



-Ar-x:oncise listing of the major references used for these 
impact studies are contained in Volume 1 of this report. 



